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Abstract. For any stationary Z"* Gibbs measure that satisfies strong spatial 
mixing, we obtain sequenees of upper and lower approximations tliat converge 
to its entropy. In the case d = 2, these approximations are efficient in the sense 
that they are accurate to within e and can be computed in time polynomial in 



1. Introduction 

The entropy of a stationary J/^ Markov random field (MRF) is notoriously dif- 
ficult to compute. Recently, Gamarnik and Katz [3] developed a technique for 
estimating entropy, and more generally pressure, for certain MRF's. Their ap- 
proach built on earlier work of Weitz [TB] who gave an algorithm for efficiently 
counting the number of independent sets in finite graphs. The algorithm was based 
on the construction of a computation tree and the proof of efficiency relied on the 
concept of strong spatial mixing (SSM) [101 Part 2, Section 2]. Coming from the 
direction of ergodic theory, we showed that a variant of the transfer matrix method 
provides efficient algorithms for estimating entropy for certain 7? MRF's [T^], [5]. 
Our argument relied on a version of SSM implied by a disagreement percolation 
condition developed in [1^ (see Proposition 14. 141 below) . We regard an algorithm 
as "efficient" if it computes upper and lower bounds accurate to within e in time 
polynomial in 1/e. 

While both approaches made use of SSM, they both required other assumptions 
as well, some involving the existence of certain kinds of periodic configurations. The 
purpose of this paper is to give approximations, using only SSM as a hypothesis, 
which estimate the entropy of 1/^ MRF's (and do so efficiently in the case d = 2). 
General sufficient conditions for SSM can be found in the literature, e.g., [2] and [T6] . 

Assuming a standard version of SSM (at exponential rate), we obtain upper 
and lower bounds that are exponentially tight (see Lemma [3.11 and Theorem 13. 2p . 
While these bounds are not explicitly computable in all cases, we believe them to 
be of independent interest. In the special case of a nearest-neighbor stationary 
Gibbs measure which satisfies SSM, we obtain an algorithm that approximates these 
bounds (Theorem 14. 5p . Combining these results yields an algorithm for approxi- 
mating entropy that is accurate to within e in time polynomial in e'^'-'-'"^*-^/'^^-'' ' ■* 
(see CoroUarv 14. 7p . Specializing to d = 2, the algorithm runs in time polynomial 
in 1/e. We also show how to modify the algorithm to approximate the pressure of 
the interaction that defines the Gibbs measure. 
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2. Background 

Wc focus on Markov random fields on the d-dimensional cubic lattice, the 

graph defined by vertex set if' and edge set {{u,w} : X]f=i 1"* — Vi\ = !}■ The 
boundary of a set S', which is denoted by 95, is tlie set of w £ Z'^ \ S* which are 
adjacent to some element of S. 

An alphabet ^ is a finite set with at least two elements. For a non-empty 
subset S C Z'', an clement u G is called a configuration; here, S is called 
the shape of u. For any configuration u with shape S and any T <Z S, denote 
hy u\t the restriction of u to T, i.e. the sub-configuration of u occupying T. For 
S,T disjoint sets, x £ and y G A^ , xy denotes the configuration on S UT 
defined by {xy)\s = x and {xy)\T = y, which we call the concatenation of x and 
y. We will sometimes informally identify a configuration a; on a shape S with the 
corresponding configuration on a translate S + v, namely the configuration y on 
S + v defined by y„ = Xu-v 

We use (J to denote the Z'' shift action on A^ defined by ((J„(a;))„ — Xu+v 
The set A^'' is a topological space when endowed with the product topology (where 
A has the discrete topology), and any subset inherits the induced topology. By a 
Z -measure, we mean a Borel probability measure on A^ . This means that any 
fi is determined by its values on the sets [w] :— {x G A^ : x\s ~ w}, where 
u) is a configuration with arbitrary finite shape S ^ Tf" . Such sets are called 
cylinder sets, and for notational convenience, rather than referring to a cylinder 
set \w\ within a measure or conditional measure, we just use the configuration w. 
For instance, \i(w^v \ u) represents the conditional measure fidw] n [v] \ [u]). A 
Z'^-measure /i is translation-invariant (or stationary) if ^{A) = ^{a^A) for all 
measurable sets A and v G Z''. A Z'^-measure is fully supported if it assigns 
strictly positive measure to every cylinder set in A^ . 

Definition 2.1. A JJ^ -measure ^ is a Markov random field (or MRF) if, 
for any finite S d 1/^ , any rj G A^ , any finite T (Z If^ s.t. dS C T C Z'' \ S*, and 
any S G A'^ with ^{S) > 0, 

(1) I S\9s) - I '5). 

Informally, fi is an MRF if, for any finite 5 C Z'', configurations on the sites in 
S and configurations on the sites in Z*^ \ (S* U dS) are /^-conditionally independent 
given a configuration on the sites in dS. In many papers, the MRF condition is 
defined in terms of a parameter r, and the set of all sites in Z'' \ S* that are within 
distance r of S* plays the role of 95*. Obviously our definition corresponds to the 
case r = 1 (a "nearest- neighbor" MRF). 

Another commonly used variant on our definition of MRF involves conditioning, 
in the right-hand side of ([T]) , on an entire configuration on Z'' \ 5 a.e. rather than 
arbitrarily large finite configurations. However, the definitions are equivalent (one 
can just take weak limits) and the finite approach is a bit more concrete. 

For two configurations y, z G A?^ on a finite set T, let £>(?/, z) = {u G Z'' : y-u ^ 
Let (i(-, •) denote the distance on Z''. 

Definition 2.2. A stationary MRF /x satisfies strong spatial mixing (SSM) 
if there exist constants C, a > 0, such that for any finite V C Z'', u £ V, dV C 
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and y, z A"'" satisfying iJ,{y), fi{z) > 0, 
\^lix \ y) ~ ^l{x \ z)\ < (7e-"^«">^^(2'^^)). 

We note that strong spatial mixing can be defined for probability measures on 
fairly arbitrary undirected graphs. Sometimes strong spatial mixing, as we have 
defined it, is called "strong spatial mixing with exponential rate." 

The following is the standard notion, in ergodic theory and information theory, 
of entropy. 

Definition 2.3. Given a "Z"^ -measure fi and a finite set S C Z'^, one defines the 
entropy o/ /i on S as: 

weA^ 

where terms with ^{w) = are omitted. 

We also have the notion of conditional entropy. 

Definition 2.4. Given a -measure ji and disjoint finite sets S,T C 11^, one 
defines the conditional entropy of [i on S , given T, as: 

where again terms with fjL{w) = are omitted. 

Let /i be a stationary Z'^-measure. The following monotonicity property is well 
known: if S,T,T' C Z'^ are finite, T' C T and 5 n T = 0, then Hf,{S \ T) < 
H^{S \ T'). We can now extend Definition 12.41 to infinite T by defining 

H,,{S I T) = limH,,{S I T„) 

for a nested sequence of finite sets Ti a T2 C . . . with UnT„ = T; by the monotonic- 
ity property just mentioned, the limit exists and does not depend on the particular 
choice of sequence T„. With this definition, it is clear that the previously mentioned 
monotonicity also holds for infinite T and T': 

Lemma 2.5. Let fi be a stationary X'^ -measure. IfS,T,T' C Z'', S is finite, T' C T 
and S r\T = , then 

H^{S I T) < H^{S I T'). 

We will find the following notation useful later. Let S and T be disjoint finite 
sets. For a stationary Z'' MRF fi and a fixed configuration y £ , with /i(y) > 0, 
we define 

Thus, we can write 

(2) H,,{S\T)^ f^ivWSly). 
If T is the disjoint union of Ti and T2, we can write 

(3) H^{S\T,UT2)= Y /^(y) E Ky^\y)H^{S \wy). 

yeA'^i: IJ.(y)>0 weA'^2: fj,(wy)>0 
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We can also define the entropy of a stationary stationary Z'^-measure itself, also 
known as entropy rate in information theory. 

Definition 2.6. The measure-theoretic entropy of a stationary "Z"^ -measure ji 
on is defined by 



= lim 



ii j2,...,id^oo jij2---jd 
where Sj-^j2...j^ denotes the x j2 x ■ • ■ x jd rectangular prism nf=i[liid- 

It is well known that the limit exists independent of the rates at which each 
ji, j2, ■ ■ ■ ,jd approach infinity [5l Theorem 15.12]. 

There is also a useful conditional entropy formula for h{p). For this, we consider 
the usual lexicographic order on Z'': a; ^ y if for some 1 < k < d, Xi = yi for 
I = 1, . . . , fc — 1 and Xk < yk- Let ~ {z E : z ^ 0}. where denotes the 
origin. 

Theorem 2.7. [Sj Equation 15.18] Let fi be a stationary -measure. Then 

hi^i) = H^{Q I V-). 

3. Entropy bounds for stationary MRF's 

Let V+ = {z e : z > 0}. Then V+ = \ . Let B„ denote the d- 
dimensional cube of side length 2n + 1 centered at 0. Let Sn = Bn H , and 

Un = Bnr\dv+. 

We claim that C/„ C dSn- To see this, note that, by definition, if x G dV^ , then 
X € 'P~ and X has a nearest neighbor y G . It follows that for some 1 < < d, 
we have Xi = yi for all i ^ k and either [xk = — 1 and yk = 0) or (a;^ = and 
yk = 1). In either case, if x G Un = Bn n 97^+, then y G i?„ and so ?/ G Sn- Thus, 
X G dSn- Figure [T] shows these sets for d = 2. 




Figure 1. Un, Sn: and dS, 
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Lemma 3.1. Let n be a stationary 11^ MRF. Then 

(4) H^{0 I dSn) < h{^L) < Hf,{0 I Un). 

Proof. Since = if/^(0 \ V^) and C/„ CV^, it follows from Lemma 1^751 that 

(5) H^iO I dSn U T'-) < < H^{0 I C/„). 

But since G Sn, Sn — and ^ is a Z'^ MRF, it follows that the left-hand 

sides of (|4]) and ([5]) agree. □ 

We remind the reader of standard notational conventions. For a function / on 
the integers, we write / = 0{n) to mean there exists a constant C > such that 
for sufficiently large n, < Cn and / = to mean there exists a constant 

C > such that for sufficiently large n, f{n) > Cn. 

Theorem 3.2. Let fi be a stationary MRF that satisfies SSM. Then 
\H^{0 I [/„)-i7^(0 I dSn)\ = e-"("). 

Proof. Let L„ = dSn \ Un- Then dSn is the disjoint union of Un and L„. For every 
configuration y G A^'^ such that ^{y) > 0, let 

E{y) = {w G A^" : niyw) > 0}. 

By ^ and ([3]), we can write 

(6) H^{0\Un)= J2 t^{y) H ^{0 \ y) a.nd 

(7) H^{0\dSn)^ J2 ^'^y) E t^{w\y)H^{Q\yw). 

y£A"'^: ti{y}>0 w£E{y) 

Fix y as above. Let C and a be the positive constants for SSM. For any config- 
uration y on C/„ and u>, w' G -ECj/), we have (i({0}, D{'w, w')) > n. By SSM applied 
toV ^ Sn, T = dSn, we have that for all x G A", y G A^", and G E{y), 

1 yw) -/^(x I yw')\ < Ce""". 

Now, 

I y) = E ^('^ I ^^^/^^^ 1 2^^)' 



and so for all w G 



ItJ-ix I ?/) - I yw)\ 



o'eEiy) 

«>'e_E(y) 

Since the function f{z) = — zlog2: is Holder continuous on [0, 1], it follows that for 
some C", a' > 0, 

\n{x I y)\ogfj.{x I y)-n{x \ yw)\ogfj.{x \ yw)\ < C"e~"'". 
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Thus, 



J2 fi{w \ y){H^{0 \ y) - H^{0 \ yw)) 



weE(y) 

< 5] I y)\H^{0 I y)-H^{0 I yw))\ 

x£A° w£E{y) 

Applying © and ([7]), we get 

\H^{0 I C/„) - H^(0 I dSn)\ < I^IC'e-"'" = e-^^("). 



□ 



By combining Lemma 13.11 and Theorem 13.21 we obtain exponentiaUy accurate 
upper and lower approximations to h{fi) for any stationary Z** MRF /i which satisfies 
SSM. In the following section, we show that when the MRF is a (nearest-neighbor) 
Gibbs measure (defined in the next section) and c? = 2, there is an efficient algorithm 
to approximate these bounds. A version of the well-known Hammersley-Clifford 
theorem [T3] shows that any fully supported (nearest-neighbor) MRF is a (nearest- 
neighbor) Gibbs measure. However, that result can fail in general; see [T] for an 
example based on a construction for finite graphs given in 

4. Computation of entropy bounds for stationary Gibbs measures 

Let J : A ^ {0, oo), (3i : A X A ^ [0, oo), i = 1, . . . , d. For a finite V C Z'^ and 
w e A^, let 

i{w)= [n 7(«))n n p^{v,v+e,). 

A configuration 6 £ A^^ is called V^-admissible if there exists at least one 
w € A^ such that I{wS) > 0. 

Definition 4.1. Given as above, for all \V\ < oo and V -admissible 5, define 
for all w E AY , 



K\w) 



The collection {A*}y,5 is called a stationary 1/'- Gibbs specification for the 
local interactions 7, f5i. 

Note that each A'' is a probability measure on A^ ^ and for U C V and it; E A^ , 

A^iw) = J2 ^\wc). 

Also, we can regard A'' as a probability measure on configurations y E A^^^^ that 
agree with S on dV. 
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Definition 4.2. A stationary Z"^ Gibbs measure for a stationary Z'* Gibbs 
specification is a stationary 11^ MRF fi on such that for any finite set V and 
S E A^^ , if ^(S) > then 6 is V-admissible and for all x G A^ 

fj.{x I S) = A''(a;). 

Specifications can be used to define MRF's, not just Gibbs measures (see [5]). 
However, we find the concept of specification most useful for Gibbs measures. 

Gibbs measures, as defined here, are often referred to as "nearest-neighbor" 
Gibbs measures in the literature. Note that since the /3i are allowed to take on the 
value 0, a Gibbs measure need not be fully supported. Also, note that, by definition, 
a necessary condition for > is ^-admissibility of 6. While there may be no 
finite procedure for determining if a configuration 6 has positive measure, there is 
a finite procedure for determining if S is T^-admissible. For this reason, we impose 
an SSM condition on the specification that defines a Gibbs measure, rather than 
the Gibbs measure itself. 

Definition 4.3. A stationary Gibbs specification A satisfies strong spatial 

mixing (SSM) if there exist constants C,a > 0, such that for all finite V C "Z,^, 
ueV,dVCTcVU dV, X G A^"\ y,z e A^ , such that 5 = y\gv and ?] = z\ov 
are V-admissible and A''(y), A''(z) > 0, then 

\K\x I y)~K'\x I z)\ < Ce-«^(M,i)fe,2)). 

Note that if the specification of a Gibbs measure /i satisfies SSM, then the mea- 
sure /i itself satisfies SSM as an MRF. It is well known that when the specification 
satisfies SSM there is a unique Gibbs measure corresponding to the specification. 
In fact, a weaker notion of spatial mixing, known as weak spatial mixing |10j . is 
sufficient. 

A simple application of the chain rule for probability distributions shows that 
our definition of SSM also implies a version for conditional distributions on larger 
sets. 

Lemma 4.4. For any stationary Z'' Gibbs specification that satisfies SSM, there 
exist constants C,a > 0, such that for any finite V C Z'', U Q V , dV C T C VUdV , 
X G AP , y,z E A^ , such that S = y\dv o^nd ij = z\gy are V-admissible and 
A^{y),A'i{z) > 0, then 

(8) |A^(:r I y) - A" (a; | 2)| < |c/|Ce-"'^(^'^(«'"». 

(The constants C, a can be taken to be those in the definition of SSM.) 

Proof. Arbitrarily order the sites in J7 as 1, 2, . . . , |C/|. Then 

It^l \ l\u\ 

K\xi I y,xi,... Xi-i) - rQ A''(a;i I z, n, . . . a;^- 



\A'{x\y)~K^{x\z)\ 



< 



\u\ i-l \ / \u\ 

^\W_K\xj\y,xi,...Xj^i)\ K'^ixj] z,xi,...Xj-i) 

1=1 Vi=i / 



Wixi I y,xi, . ..Xi-i) - t<^{xi I z,xi, . . .Xi-i)\ 



< C\U\e~°"^'^'^'-^'^^'''^\ 



8 



BRIAN MARCUS AND RONNIE PAVLOV 



□ 



The following is the main result of this section. 

Theorem 4.5. Let ji he a stationary Gibbs measure whose specification satisfies 
SSM. Let (Kn), n e N, 6e a sequence of sets satisfying Kn C i?„ and \Kn\ = 
0(n''-i). Then there is an algorithm which, on input n, computes upper and lower 

bounds to H^{0 \ Kn) in time e*^*-"* ' ' to within tolerance e~" 

Remark 4.6. For this and all subsequent results involving running time of algo- 
rithms involving /i, we do not count computation of the Gibbs parameters 7 and Pi 
towards the claimed running time. (In other words, we assume that we are given 
approximations to 7 and Pi with arbitrarily good precision before performing any 
computation.) We also note that the algorithms here do not depend on knowledge 
of specific values of the parameters C and a of SSM. 

As an immediate consequence of Lemma 13.11 Theorem 13.21 and Theorem 14.51 
(applied to Kn = dSn-i and Kn = C/„), we have: 

Corollary 4.7. Let n be a stationary 1^ Gibbs measure whose specification satisfies 
SSM. Then there is an algorithm which, on input n, computes upper and lower 

bounds to h{^) in time e*^^"* ' ' to within tolerance e~^^^"' . 

Note that for d = 2 this gives an algorithm to compute ft,(/.i) to within 0(l/n) 
in polynomial time (in n). 

For the proof of Theorem 14.51 we will need the following result. 

Lemma 4.8. Let ^ be a stationary 1^ Gibbs measure. Let (Kn), n G N, be a 
sequence of sets satisfying Kn C i3„ and \Kn\ ~ 0{n'^^^). Then for any sequence 
(™n)neN of positive integers, there is an algorithm which, on input n, determines 
which 5 G ^9S"+m„ Q^j-p Bn+m„ -admissible and, for those which are, computes 
K^iw) for all w G A^" , in running time e'^^^""'"™") \ 

Proof. For simplicity, we prove this only for d = 2. The general case follows along 
similar lines. 

Fix sequences (Kn) and (m„), a particular value of n, and w and 6 as in the 
statement of the theorem. 



L^{w) := J2 I{wc5). 



We will show that 
(9) l'{w) = L{5)x[ n 



n+TTiTi — 1 



where each Afi is a square matrix and x, y are vectors, all indexed by ^ " m„,n+m„] _ 

For a e ^[-"-™n,n+m„]^ .^j.-^^ ^ ^ a-n-rn„, ■ ■ ■ a„+m„ ■ 
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For i = —n — m„, . . . , n + m„ — 1, define the transfer matrix 



\j — — n~mn J \j ——n—7n^ 



/32(l5i,-n-rn„-l, a-n-m„ )/32 (ln+m„ , <5i,n+7Ti„ + l ) 

Let Vi = {i} X [— n — 7ti„ + 1, . . . n + m„ — 1] and let 

{^U){a,b) = (M)(a.b) 

except when Vi n Kn / and a|{j:(i.j)eK„} 7^ w|\/in/f„, in which case we set 
(M,)(a,fc) = 0. Let 

^« ^ n /5i(^i,-n-m^-i,ai), and let 



J]^ /3i(aj,(5„+™„+ij))7(aj) 

J — —n — 771,1 

{^n+7nn,—n — 7nn — l ; ^ — n — 771,1 )/^2 ('^77 + 777„ ; f^77 + 7n„ ,n+77i„ + 1 ) • 

The reader can now verify (jH). 

Note that each Mi can be constructed in time (^e'^^"^™"-') = QO{n+m„) ^ ^ ^^-^^j^ y 
can be computed in time ^Oin+m^) ^ j^j-j -^q computed in time 0{n + m„). 
Each matrix muhiphcation takes time at most {QO{n+7nn)y^ _ gO(7i+77t„)^ Thus, 
I^{'w) can be computed in time gO{n+mr,) ^ This can be done for all w G in 

time QO{n+mn) ^0{n) _ ^0{n+m„) 

Since 

we can compute A^{w) for all w £ ^^"^ and all i3„+m„ -admissible i5 G 
time (eO("+™"))^ gO(n+m„)_ 

For d > 2, the proof follows along similar lines using transfer matrices indexed 
by configurations on (d — l)-dimensional arrays. 

□ 

Proposition 4.9. Let ji he a stationary If^ Gibbs measure whose specification sat- 
isfies SSM with constants C and a. Let {Kn), n £N, be a sequence of sets satisfying 
Kn C Bn and \Kn\ = 0(n'^~^). Then for any sequence (nin) of positive integers, 
there is an algorithm which, on input n, computes upper and lower bounds 
and (w) to ^{w), for all w G , in time eO((n+?n„) ) ^ such that 

11+ {w) - fr{w) < Ce-°'"'"\Kn\. 

Proof. Fix sequences (Kn) and (m„), a particular value of n, and w as in the 
statement of the theorem. Observe that 
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Let S"^ be a configuration S which achieves max^^ ^ -admissible s} ^^i'^) ^^'^ 
let Sw be a configuration S which achieves min^^ ^ -admissible s} ^^{^)- Since 
strict positivity of implies i?„+„i^ -admissibility, it follows that 

Since ^ satisfies SSM, it follows by Lemma 14.41 (applied to V = Bn+m„ , T — dV 
and U ~ Kn) that 

(10) < A*"(u;) - A^^-{w) < Ce-"™"|A'„|. 

By Lemma |4.8[ we can identify all i3m„+Ti-admissible 6 and compute A^{w) for 
all such S and ah w G A'^" in time eO(("+™")''"'). Thus in time eC'(("+'"")''"') we 
can identify, for all w E , Si^ , and (5™ and compute the upper and lower bounds 
A^- (w) and A''" 

This, together with ([10]), completes the proof. □ 
Similarly, we have: 

Proposition 4.10. Let ^ he a stationary 1/^ Gibbs measure whose specification 
satisfies SSM with constants C and a. Let [Kn), n £ N, he a sequence of sets 
satisfying Kn C Bn \ {0} and \Kn\ = 0{n'^~^). Then for any sequence (rrin) of 
positive integers, there is an algorithm which, on input n, computes upper and lower 
bounds /i^(a;o | w) and H^{xo \ w) to /i(a;o | w) for all xq £ A and w G A^'^ with 
^l{w) >0 in time eO(("+™")''"') such that 

^i+{xo I w) - ^i-ixo I w) < Ce""™". 

Proof. Fix sequences (Kn) and (m„), a particular value of n, and w as in the 
statement of the theorem. Write 

yu(xo \ w) = ^ n{xo I w,d)^i{S I w). 

As in the proof of Proposition 14.91 we can find _B„+m^ -admissible S^"'"^ and S^g^w 
such that 

AS^o.^-(^xq\w) < fi{xo I w) < A^""'" {xq\w) and 

< A^^°" {xo\w) ~ A^^o.^i^x^lu}) < Ce^"™". 

(here, we apply SSM to V ^ B„+,n„, T ^ {dV) U K^, U ^ {0}). Then ap- 
ply Lemma to compute these bounds, i.e., compute A"^ (xqw), A* "' {w), 
A''-o."'(xow), and A''-o."'(iy). □ 

Proof of Theorem \4-5\ Let {Kn), n G N, be a sequence of sets satisfying Kn C Bn 
and \Kn\ = 0{n'^~^). We will first describe how to compute upper and lower 
bounds for arbitrary choice of (m„), and then describe how to choose the proper 
values (m„) for our algorithm. 

For any n and m„, let /i+(ii;), /i^ (w), /i+(a;o|w), /i^ (a;o|w) be as in Proposi- 
tions HH and IHTOl 

Let f{x) = — xlogcc. Let {xq \ w) denote whichever of /x"'' (xq | w),^~{xo \ w) 
achieves min (/(/^^(xo | w)), f{^~ {xq \ w))). Since / is concave and Holder con- 
tinuous on [0, 1], for some C',a' > (independent of n and to„) we have 

(11) < /(/x(xo I w)) - fifi—ixo \ w)) < C'e-"'™". 
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Recall that 

Let H^{0 I i^„) denote the expression obtained by substituting n~(w) for fi{w) 
and (a;o | w) for /x(x'o | u;): 

H;{0 I i^„) = ^ fi-{w) J2 /(m""(2^o I ^)). 

weA^'^ xo£A° 

TheniJ-(0 I A'„) <if^(0 | X„). 

Now, we estimate the difference between i?^(0 | A'„) and H^{0 \ K^)- Using 
(fTT|) . we see that 



(12) iy^(o I j^„)-ff-(o I if„) 

= E E (/(/^(^o I w)) -/(pi~"(xo I w))) 

+ E (mH-m"(w')) E /(M~"(a;o I w)) 

for some constant rj (depending on the growth rate of |A'„|). The reader can check 
that there then exists a constant L so that for every n, if to„ > Ln'^~^, then 
H^iO I K„) - i?-(0 I < 0.5e-"'"\ 

We also note that the computation time of H^{0 \ Kn) is e'^'("+'"") ' (the 
total amount of time to compute fi^{w) and /(/i (xq | w)) for all w G and 
xa e A°.) 

For the upper bound, let /i++(xo | w) be whichever of /^^(a;o | 'w),^~{xq \ w) 
achieves max (/(^^(xo | w)),f{fi^{xo \ w))) if x,y < 1/e or x,y > 1/e, and 1/e 
otherwise. Using Holder continuity of /, as well as the fact that f{x) achieves its 
maximum at x = 1/e, we have: 

(13) < fi^l++ixo I w)) - /(/x(xo I w)) < C'e-"'"". 
Then 

H+{0 I A'„) = /^^(^) E /(/^^^(^o I ^«)) 

is an upper bound for iJ^(0 | A'„). 
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Using (fT3|). we see that 
(14) H+{0\K^)-H^{0\K^) 

= E E /(/^""^(^o I - E /^(^) E /(/^(^o I ^)) 

wGA^" 2:06.40 weA^" xoeA° 

^g^K„ 2:06.4° 

•iu6^-^'" 2:06.4° 

For every n, if m„ > Ln"^^^ (the L is the same as for the lower bound), then 
H+{0 I Kn) ~ H^{0 I K^) < O.Se-"""'. The time to compute H+{0 \ K^) is 
gO((n+m„)''-i)^ the same as for iJ-(0 | A'„). 

We now describe the algorithm for choosing the values (m„). We note that 
without knowledge of the explicit constants C and a from the strong spatial mixing 
of /X, we cannot explicitly compute the constant L. However, for our purposes, 
knowledge of L is unnecessary. 

The algorithm uses parameters n and j which both start off equal to 1, though 
they will be incremented later. The algorithm consists of one main loop which is 
run repeatedly. At the beginning of the loop, the above bounds H^{0 \ Kn) and 

H^{0 I Kn) are computed for m„ = jn'^~^ . If the bounds are not within e~" of 
each other, then j is incremented by 1 and the algorithm returns to the beginning 
of the loop. When the bounds are within e~" of each other (which will happen 
for large enough j by the comments following (fT2|) and ([T4|) ). then m„ is defined 
to be jn'^~^, the value of n is incremented by 1, and the algorithm returns to the 
beginning of the loop. 

By the above discussion, there exists L so that j will never be incremented beyond 
L in this algorithm. This means that there exists J so that for all sufficiently large 
n, TO„ = Jn'^^^. Therefore, for all n, the algorithm yields upper and lower bounds 
to within tolerance e""'' ^ in time eO({n+jn'' ^)'^ ^) _ ^o{n'-'' )_ 

□ 

Remark 4.11. We remark that the algorithms in Propositions 14. 91 and 14. lOl can be 
simplified if one uses knowledge of specific values of the constants C and a in the 
definition of SSM. Namely, one can compute A^{w) (or A'^(xo | w)) for any fixed 
Sn+m„-admissible configuration S and then set the upper and lower bounds fi^{w) 
(or iJ,^\xo I w)) to be A^{w) ± Ce-"""|iv:„| (or A^{xo \ w) ± Ce""™"). 

In theory, we can also dispense with the auxiliary sequence (m„) in Propo- 
sition 14.101 we could instead bound fi{xo \ w) by the minimum and maximum 
possible values of n{xo \ w,d) for configurations S on dBn, which would give ap- 
proximations of tolerance Ce~"" in time 0(e" ). A similar simplification could 
be done for Proposition l4.9l as well, but it would not be useful for our proof of Theo- 
rem l4.5l note that in formula ([T2|) . the upper bound on /i(w) — /x^(u') is multiplied 
by and so this upper bound must be at most e~^^(" \ Therefore, the 

described simplification for Proposition 14.101 would not reduce the overall order of 
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computation time for Theorem I4.5| since the algorithm from Proposition l4. 91 would 

still require time e'^^"*'' ^' ^ . 

Finally, we note that in Proposition 14. 101 when Kn = dSn-i, there is no need to 
bound the conditional probabilities fJ.{xo | w), as they can be computed exactly (by 
using the methods of Lemma [4. 8p . 

Wc will now describe how to extend Theorem 14.51 and Corollary 14.71 to give 
bounds for pressure in addition to entropy. Given local interactions 7, /3i, define: 

X = {x e A^"" : for all v £ Z'' and I < i < d, /3i (a;„ , x„+e J > 0}. 

X is the set of configurations on Z'' defined by nearest-neighbor constraints and so 
belongs to the class of nearest-neighbor (or 1-step) shifts of finite type [5] . 
Let / : X ^ M be defined by 

d 

(15) /(.t) log7(.To) + \ogP,{xo,Xe,). 

i=l 

Definition 4.12. Let X and f be as above. Define the pressure of f by 

P{f) = max h{u) + j fdu, 

where the max is taken over all stationary measures v with support contained in 
X . A measure which achieves the max is called an equilibrium state for f. 

Alternatively, pressure can be defined directly in terms of X and /, without 
reference to stationary measures. The definition of pressure which we have used 
is a corollary of the well-known variational principle [ITl Chapter 9]. For general 
dynamical systems, the max is merely a sup; however, in our context, the sup is 
always achieved. 

It is well known that any equilibrium state for / is a Gibbs measure for the 
specification defined by the interactions 7, /?i [TSl Chapter 4]. As mentioned earlier, 
when the specification satisfies SSM, there is only one Gibbs measure /i that satisfies 
that specification, and so ^ is an (unique) equilibrium state for /. 

Corollary 4.13. Let j^fii be local interactions which define a stationary Gibbs 
specification that satisfies SSM. Let f be as in 1115\). Then there is an algorithm 

to compute upper and lower bounds to P{f) in time e'^'-"' * ^ to within tolerance 

g-n(n)^ 

Proof. Let /i be the unique Gibbs measure that satisfies the specification. Then 
Corollar V 14.71 applies to compute such bounds for h{ii). 

It follows from Proposition 14.91 that for any configuration w on a single site or 
edge of Z'', one can compute upper and lower bounds to ijl{w) in time e*^*-" ^ to 
within tolerance e~^*^"^ (in fact, this follows easily from weak spatial mixing). Thus 
one can compute upper and lower bounds to J fdfjL in the same time with the same 
tolerance. 

Finally, recall that /i is an equilibrium state since its specification satisfies SSM, 
and so we can compute the desired bounds for /i(/.i) + J /d/i = P{f )- 

□ 



There are a variety of conditions in the literature which guarantee SSM of an 
MRF: for example, see [2], [4], [6], [M], [16], and [H]. We present the one from 



14 



BRIAN MARCUS AND RONNIE PAVLOV 



[l6] here as one of the most general and easy to state. Let A be a stationary Gibbs 
specification. Let 

q{A)= max d{Ay,A'), 

where d denotes total variation distance of distributions on A°. Let pc ~ Pc{'^'^) 
denote the critical probability for site percolation in Z'^. (We will not define Pc(Z'^) 
or discuss percolation theory here; for a good introduction to the subject, see [7].) 

Proposition 4.14. If q{h) < Pc, then A satisfies SSM. 

This result is essentially contained in [lU Theorem 1]; see [U Theorem 3.10] for 
more explanation. 
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